Methods that integrate visible light eye tracking in scanning laser projection displays

ABSTRACT

Systems, devices, and methods that integrate eye tracking capability into scanning laser projector (“SLP”)-based wearable heads-up displays are described. At least one narrow waveband laser diode is used in an SLP to define one or more portion(s) of a visible image. At least one corresponding narrow waveband photodetector is aligned to detect reflections of the portion(s) of the image from features of the eye. A holographic optical element (“HOE”) may be used to combine the image and environmental light into the user&#39;s “field of view.” Three narrow waveband photodetectors each responsive to a respective one of three narrow wavebands output by the RGB laser diodes of an RGB SLP are aligned to detect reflections of a projected RGB image from features of the eye.

BACKGROUND Technical Field

The present systems, devices, and methods generally relate to scanninglaser-based eye tracking technologies and particularly relate tointegrating eye tracking functionality into a scanning laser projectorbased wearable heads-up display.

Description of the Related Art Wearable Heads-Up Displays

A head-mounted display is an electronic device that is worn on a user'shead and, when so worn, secures at least one electronic display within aviewable field of at least one of the user's eyes, regardless of theposition or orientation of the user's head. A wearable heads-up displayis a head-mounted display that enables the user to see displayed contentbut also does not prevent the user from being able to see their externalenvironment. The “display” component of a wearable heads-up display iseither transparent or at a periphery of the user's field of view so thatit does not completely block the user from being able to see theirexternal environment. Examples of wearable heads-up displays include:the Google Glass®, the Optinvent Ora®, the Epson Moverio®, and the SonyGlasstron®, just to name a few.

The optical performance of a wearable heads-up display is an importantfactor in its design. When it comes to face-worn devices, however, usersalso care a lot about aesthetics. This is clearly highlighted by theimmensity of the eyeglass (including sunglass) frame industry.Independent of their performance limitations, many of the aforementionedexamples of wearable heads-up displays have struggled to find tractionin consumer markets because, at least in part, they lack fashion appeal.Most wearable heads-up displays presented to date employ large displaycomponents and, as a result, most wearable heads-up displays presentedto date are considerably bulkier and less stylish than conventionaleyeglass frames.

A challenge in the design of wearable heads-up displays is to minimizethe bulk of the face-worn apparatus will still providing displayedcontent with sufficient visual quality. There is a need in the art forwearable heads-up displays of more aesthetically-appealing design thatare capable of providing high-quality images to the user withoutlimiting the user's ability to see their external environment.

Eye Tracking

Eye tracking is a process by which the position, orientation, and/ormotion of the eye may be measured, detected, sensed, determined(collectively, “measured”), and/or monitored. The position, orientation,and/or motion of the eye may be measured in a variety of different ways,the least invasive of which typically employ one or more opticalsensor(s) (e.g., cameras) to optically track the eye. Common techniquesinvolve illuminating or flooding the entire eye, all at once, withinfrared light and measuring reflections with at least one opticalsensor that is tuned to be sensitive to the infrared light. Informationabout how the infrared light is reflected from the eye is analyzed todetermine the position(s), orientation(s), and/or motion(s) of one ormore eye feature(s) such as the cornea, pupil, iris, and/or retinalblood vessels.

Eye tracking functionality is highly advantageous in applications ofwearable heads-up displays. Some examples of the utility of eye trackingin wearable heads-up displays include: influencing where content isdisplayed in the user's field of view, conserving power by notdisplaying content that is outside of the user's field of view,influencing what content is displayed to the user, determining where theuser is looking, determining whether the user is looking at displayedcontent on the display or through the display at their externalenvironment, and providing a means through which the user maycontrol/interact with displayed content. However, incorporating eyetracking functionality in a wearable heads-up display conventionallyadds unwanted bulk to the system. Eye tracking systems available todaygenerally implement multiple dedicated components with very stringentpositioning requirements which undesirably increase the overall size andform factor of the system when incorporated into a wearable heads-updisplay. There is a need in the art for systems, devices, and methods ofeye tracking that can integrate into wearable heads-up displays withminimal effect on the size and form factor of the system.

BRIEF SUMMARY

A laser projector with an integrated eye tracker may be summarized asincluding: a laser module including an infrared laser diode to output aninfrared light and at least one visible light laser diode to output avisible light; a scan mirror aligned with an output of the laser moduleto receive both the infrared light and the visible light and tocontrollably reflect both the infrared light and the visible light; awavelength-multiplexed holographic optical element aligned to receiveboth the infrared light and the visible light reflected from the scanmirror and to redirect both the infrared light and the visible lighttowards an eye of a user, wherein the wavelength-multiplexed holographicoptical element includes a first hologram that is responsive to thevisible light and unresponsive to the infrared light and a secondhologram that is responsive to the infrared light and unresponsive tothe visible light; and an infrared detector aligned to receive at leasta portion of infrared light reflected from the eye of the user. Thewavelength-multiplexed holographic optical element may comprise at leasttwo distinct layers of holographic material, a first layer ofholographic material that includes the first hologram and a second layerof holographic material that includes the second hologram.Alternatively, the wavelength-multiplexed holographic optical elementmay comprise a single volume of holographic material that includes boththe first hologram and the second hologram. The at least one visiblelight laser diode in the laser module may include at least one visiblelight laser diode selected from the group consisting of: a red laserdiode, a green laser diode, a blue laser diode, and any combination of ared laser diode, a green laser diode, and/or a blue laser diode.

The first hologram may apply a first optical power to the visible lightand the second hologram may apply a second optical power to the infraredlight, the second optical power different from the first optical power.The first optical power may be a positive optical power and the firstoptical power may be greater than the second optical power. The secondoptical power may be less than or equal to zero.

The laser projector may further include: a support frame that has ageneral shape and geometry of a pair of eyeglasses, wherein the lasermodule, the scan mirror, the wavelength-multiplexed holographic opticalelement, and the infrared detector are all carried by the support frame,and wherein the wavelength-multiplexed holographic optical element issubstantially transparent to environmental light and positioned in afield of view of at least one eye of the user when the support frame isworn on a head of the user.

A wearable heads-up display may be summarized as including: a supportframe that in use is worn on a head of a user; a laser module carried bythe support frame, the laser module including an infrared laser diode tooutput an infrared light and at least one visible light laser diode tooutput a visible light; a scan mirror carried by the support frame andaligned with an output of the laser module to receive both the infraredlight and the visible light output by the laser module, the scan mirrorto controllably reflect both the infrared light and the visible light; awavelength-multiplexed holographic optical element carried by thesupport frame and positioned within a field of view of at least one eyeof the user when the support frame is worn on the head of the user, thewavelength-multiplexed holographic optical element aligned to receiveboth the infrared light and the visible light reflected from the scanmirror and to redirect both the infrared light and the visible lighttowards the at least one eye of the user when the support frame is wornon the head of the user, wherein the wavelength-multiplexed holographicoptical element includes a first hologram that is responsive to thevisible light and unresponsive to the infrared light and a secondhologram that is responsive to the infrared light and unresponsive tothe visible light, and wherein the wavelength-multiplexed holographicoptical element is substantially transparent to environmental light; andan infrared detector carried by the support frame and aligned to receiveat least a portion of infrared light reflected from the at least one eyeof the user when the support frame is worn on the head of the user. Thesupport frame may have a general shape and appearance of a pair ofeyeglasses. The wavelength-multiplexed holographic optical element maycomprise at least two distinct layers of holographic material, a firstlayer of holographic material that includes the first hologram and asecond layer of holographic material that includes the second hologram.Alternatively, the wavelength-multiplexed holographic optical elementmay comprise a single volume of holographic material that includes boththe first hologram and the second hologram. The at least one visiblelight laser diode in the laser module may include at least one visiblelight laser diode selected from the group consisting of: a red laserdiode, a green laser diode, a blue laser diode, and any combination of ared laser diode, a green laser diode, and/or a blue laser diode.

The first hologram may apply a first optical power to the visible lightand the second hologram may apply a second optical power to the infraredlight, the second optical power different from the first optical power.The first optical power may be a positive optical power and the firstoptical power may be greater than the second optical power. The secondoptical power may be less than or equal to zero.

A method of operating a laser projector to project an image to an eye ofa user and to track the eye of the user may be summarized as including:outputting visible light by at least a first laser diode of the laserprojector, the visible light representative of at least a portion of theimage; outputting infrared light by an infrared laser diode of the laserprojector; controllably and reflectively scanning both the visible lightand the infrared light by a scan mirror of the laser projector;redirecting both the visible light and the infrared light towards theeye of the user by a wavelength-multiplexed holographic optical element;detecting a reflection of at least a portion of the infrared light fromthe eye of the user by an infrared photodetector; and determining aposition of at least one feature of the eye based on the reflection ofat least a portion of the infrared light from the eye of the userdetected by the infrared photodetector.

Redirecting both the visible light and the infrared light towards theeye of the user by a wavelength-multiplexed holographic optical elementmay include: applying a first optical power to the visible light by afirst hologram of the wavelength-multiplexed holographic opticalelement; and applying a second optical power to the infrared light by asecond hologram of the wavelength-multiplexed holographic opticalelement, the second optical power different from the first opticalpower. Applying a first optical power to the visible light by a firsthologram of the wavelength-multiplexed holographic optical element mayinclude applying a first positive optical power to the visible light bythe first hologram of the wavelength-multiplexed holographic opticalelement. Applying a second optical power to the infrared light by asecond hologram of the wavelength-multiplexed holographic opticalelement may include applying a second optical power that is less thanthe first optical power to the infrared light by the second hologram ofthe wavelength-multiplexed holographic optical element. Applying asecond optical power that is less than the first optical power to theinfrared light by the second hologram of the wavelength-multiplexedholographic optical element may include applying a second optical powerthat is less than or equal to zero to the infrared light by the secondhologram of the wavelength-multiplexed holographic optical element.Determining a position of at least one feature of the eye based on thereflection of the infrared light from the eye of the user detected bythe infrared photodetector may include determining the position of atleast one feature of the eye based on the reflection of the infraredlight from the eye of the user by a processor. Outputting visible lightby at least a first laser diode of the laser projector may include atleast one of: outputting red light by a red laser diode of the laserprojector; outputting green light by a green laser diode of the laserprojector; and/or outputting blue light by a blue laser diode of thelaser projector.

A laser projector with an integrated eye tracker may be summarized asincluding: a laser module to output laser light, wherein the lasermodule includes a first laser diode to contribute a first visible laserlight in a first narrow waveband to the laser light output by the lasermodule, the first visible laser light representative of at least a firstportion of an image; a scan mirror aligned with an output of the lasermodule to receive the laser light output by the laser module and tocontrollably reflect the laser light output by the laser module; aholographic optical element aligned to receive the laser light reflectedfrom the scan mirror and to redirect the laser light towards an eye of auser, wherein the holographic optical element includes a first hologramthat is responsive to the first visible laser light in the first narrowwaveband and unresponsive to light that is outside of the first narrowwaveband; and a first narrow waveband photodetector aligned to receiveat least a portion of the laser light that is reflected from the eye ofthe user, wherein the first narrow waveband photodetector is responsiveto the first visible laser light in the first narrow waveband andunresponsive to light that is outside of the first narrow waveband. Thelaser module may include a second laser diode to contribute a secondvisible laser light in a second narrow waveband to the laser lightoutput by the laser module, the second narrow waveband different fromthe first narrow waveband. The second visible laser light may berepresentative of at least a second portion of the image. The firsthologram may be unresponsive to the second visible laser light in thesecond narrow waveband. The holographic optical element may be awavelength-multiplexed holographic optical element that includes atleast a second hologram that is responsive to the second visible laserlight in the second narrow waveband and unresponsive to the firstvisible laser light in the first narrow waveband. The laser projectormay further include a second narrow waveband photodetector, the secondnarrow waveband photodetector responsive to the second visible laserlight in the second narrow waveband and unresponsive to light that isoutside of the second narrow waveband. The laser module may include athird laser diode to contribute a third visible laser light in a thirdnarrow waveband to the laser light output by the laser module, the thirdnarrow waveband different from both the first narrow waveband and thesecond narrow waveband. The third visible laser light may berepresentative of at least a third portion of the image. The firsthologram may be unresponsive to third visible laser light in the thirdnarrow waveband. The second hologram may be unresponsive to the thirdvisible laser light in the third narrow waveband. Thewavelength-multiplexed holographic optical element may include a thirdhologram that is responsive to the third visible laser light in thethird narrow waveband and unresponsive to both the first visible laserlight in the first narrow waveband and the second visible laser light inthe second narrow waveband. The laser projector may further include athird narrow waveband photodetector, the third narrow wavebandphotodetector responsive to the third visible laser light in the thirdnarrow waveband and unresponsive to light that is outside of the thirdnarrow waveband.

The first laser diode may be a red laser diode and the first narrowwaveband may correspond to a first range of wavelengths that are visibleas red to the eye of the user. The second laser diode may be a greenlaser diode and the second narrow waveband may correspond to a secondrange of wavelengths that are visible as green to the eye of the user.The third laser diode may be a blue laser diode and the third narrowwaveband may correspond to a third range of wavelengths that are visibleas blue to the eye of the user.

The holographic optical element may comprise at least three distinctlayers of holographic material: a first layer of holographic materialthat includes the first hologram, a second layer of holographic materialthat includes the second hologram, and a third layer of holographicmaterial that includes the third hologram. Alternatively, theholographic optical element may comprise a single volume of holographicmaterial that includes all three of the first hologram, the secondhologram, and the third hologram.

The first hologram may apply a first optical power to the first visiblelaser light in the first narrow waveband, the second hologram may applythe same first optical power to the second visible light in the secondnarrow waveband, and the third hologram may apply the same first opticalpower to the third visible laser light in the third narrow waveband.

The laser projector may further include: a support frame that has ageneral shape and appearance of a pair of eyeglasses, wherein the lasermodule, the scan mirror, the wavelength-multiplexed holographic opticalelement, and the first narrow waveband photodetector are all carried bythe support frame, and wherein the wavelength-multiplexed holographicoptical element is substantially transparent to environmental light andpositioned in a field of view of at least one eye of the user when thesupport frame is worn on a head of the user.

A wearable heads-up display may be summarized as including: a supportframe that in use is worn on a head of a user; a laser module carried bythe support frame, the laser module including a first laser diode tooutput a first visible laser light in a first narrow waveband, the firstvisible laser light representative of at least a first portion of animage; a scan mirror carried by the support frame and aligned with anoutput of the laser module to receive the first visible laser light andto controllably reflect the first visible laser light; a holographicoptical element carried by the support frame and positioned within afield of view of at least one eye of the user when the support frame isworn on the head of the user, the holographic optical element aligned toreceive the first visible laser light reflected from the scan mirror andto redirect the first visible laser light towards the at least one eyeof the user when the support frame is worn on the head of the user,wherein the holographic optical element includes a first hologram thatis responsive to the first visible laser light in the first narrowwaveband and unresponsive to light that is outside of the first narrowwaveband, and wherein the holographic optical element is substantiallytransparent to environmental light; and a first narrow wavebandphotodetector carried by the support frame and aligned to receive atleast a portion of the first visible laser light that is reflected fromthe at least one eye of the user when the support frame is worn on thehead of the user, wherein the first narrow waveband photodetector isresponsive to the first visible laser light in the first narrow wavebandand unresponsive to light that is outside of the first narrow waveband.The support frame may have a general shape and appearance of a pair ofeyeglasses.

The laser module of the wearable heads-up display may include a secondlaser diode to output a second visible laser light in a second narrowwaveband, the second narrow waveband different from the first narrowwaveband, wherein the second visible laser light is representative of atleast a second portion of the image. The first hologram may beunresponsive to the second visible laser light in the second narrowwaveband. The holographic optical element may be awavelength-multiplexed holographic optical element that includes atleast a second hologram that is responsive to the second visible laserlight in the second narrow waveband and unresponsive to the firstvisible laser light in the first narrow waveband. The wearable heads-updisplay may further include a second narrow waveband photodetector, thesecond narrow waveband photodetector responsive to the second visiblelaser light in the second narrow waveband and unresponsive to light thatis outside of the second narrow waveband. The laser module of thewearable heads-up display may include a third laser diode to output athird visible laser light in a third narrow waveband, the third narrowwaveband different from both the first narrow waveband and the secondnarrow waveband, wherein the third visible laser light is representativeof at least a third portion of the image. The first hologram may beunresponsive to third visible laser light in the third narrow waveband.The second hologram may be unresponsive to the third visible laser lightin the third narrow waveband. The wavelength-multiplexed holographicoptical element may include a third hologram that is responsive to thethird visible laser light in the third narrow waveband and unresponsiveto both the first visible laser light in the first narrow waveband andthe second visible laser light in the second narrow waveband. Thewearable heads-up display may further include a third narrow wavebandphotodetector, the third narrow waveband photodetector responsive to thethird visible laser light in the third narrow waveband and unresponsiveto light that is outside of the third narrow waveband.

The first laser diode may be a red laser diode and the first narrowwaveband may correspond to a first range of wavelengths that are visibleas red to the eye of the user. The second laser diode may be a greenlaser diode and the second narrow waveband may correspond to a secondrange of wavelengths that are visible as green to the eye of the user.The third laser diode may be a blue laser diode and the third narrowwaveband may correspond to a third range of wavelengths that are visibleas blue to the eye of the user.

The holographic optical element of the wearable heads-up display maycomprise at least three distinct layers of holographic material: a firstlayer of holographic material that includes the first hologram, a secondlayer of holographic material that includes the second hologram, and athird layer of holographic material that includes the third hologram.Alternatively, the holographic optical element of the wearable heads-updisplay may comprise a single volume of holographic material thatincludes all three of the first hologram, the second hologram, and thethird hologram.

The first hologram may apply a first optical power to the first visiblelaser light in the first narrow waveband, the second hologram may applythe same first optical power to the second visible light in the secondnarrow waveband, and the third hologram may apply the same first opticalpower to the third visible laser light in the third narrow waveband.

A method of operating a laser projector to project an image to an eye ofa user and to track the eye of the user may be summarized as including:outputting visible laser light by a laser module, wherein the lasermodule includes at least a first laser diode and outputting visiblelaser light by the laser module includes outputting a first visiblelaser light in a first narrow waveband from the first laser diode of thelaser module, and wherein the first visible laser light isrepresentative of at least a first portion of an image; controllably andreflectively scanning the visible laser light by a scan mirror;redirecting the visible laser light towards the eye of the user by aholographic optical element; detecting a reflection of at least aportion of the visible laser light from the eye of the user by at leasta first narrow waveband photodetector, wherein the first narrow wavebandphotodetector is responsive to light in the first narrow waveband andsubstantially unresponsive to light that is outside of the first narrowwaveband, and wherein detecting a reflection of the at least a portionof the visible laser light from the eye of the user by at least a firstnarrow waveband photodetector includes detecting a reflection of thefirst portion of the image by the first narrow waveband photodetector;and determining a position of at least one feature of the eye based onat least the reflection of the first portion of the image from the eyeof the user detected by the first narrow waveband photodetector.

The laser module may include a second laser diode and outputting visiblelight by a laser module may further include outputting a second visiblelaser light in a second narrow waveband by the second laser diode of thelaser module, the second narrow waveband different from the first narrowwaveband. The second visible laser light may be representative of atleast a second portion of the image. Controllably and reflectivelyscanning the visible laser light by a scan mirror may includecontrollably and reflectively scanning both the first portion of theimage and the second portion of the image by the scan mirror. Theholographic optical element may be a wavelength-multiplexed holographicoptical element comprising a first hologram that is responsive to lightin the first narrow waveband and unresponsive to light that is outsidethe first narrow waveband and a second hologram that is responsive tolight in the second narrow waveband and unresponsive to light that isoutside the second narrow waveband. Redirecting the visible laser lighttowards the eye of the user by the holographic optical element mayinclude redirecting the first portion of the image towards the eye ofthe user by the first hologram of the wavelength-multiplexed holographicoptical element and redirecting the second portion of the image towardsthe eye of the user by the second hologram of the wavelength-multiplexedholographic optical element. Detecting a reflection of at least aportion of the visible laser light from the eye of the user by at leasta first narrow waveband photodetector may further include detecting areflection of the second portion of the image from the eye of the userby a second narrow waveband photodetector, wherein the second narrowwaveband photodetector is responsive to light in the second narrowwaveband and substantially unresponsive to light that is outside of thesecond narrow waveband. Determining a position of at least one featureof the eye based on at least the reflection of the first portion of theimage from the eye of the user detected by the first narrow wavebandphotodetector may further include determining a position of at least onefeature of the eye based on the reflection of the second portion of theimage from the eye of the user detected by the second narrow wavebandphotodetector.

The laser module may include a third laser diode and outputting visiblelight by a laser module may further include outputting a third visiblelaser light in a third narrow waveband by the third laser diode of thelaser module, the third narrow waveband different from both the firstnarrow waveband and the second narrow waveband. The third visible laserlight may be representative of at least a third portion of the image.Controllably and reflectively scanning the visible laser light by a scanmirror may further include controllably and reflectively scanning thethird portion of the image by the scan mirror. Thewavelength-multiplexed holographic optical element may further include athird hologram that is responsive to light in the third narrow wavebandand unresponsive to light that is outside the third narrow waveband.Redirecting the visible laser light towards the eye of the user by theHOE may further include redirecting the third portion of the imagetowards the eye of the user by the third hologram of thewavelength-multiplexed holographic optical element. Detecting areflection of at least a portion of the visible laser light from the eyeof the user by at least a first narrow waveband photodetector mayfurther include detecting a reflection of the third portion of the imagefrom the eye of the user by a third narrow waveband photodetector,wherein the third narrow waveband photodetector is responsive to lightin the third narrow waveband and substantially unresponsive to lightthat is outside of the third narrow waveband. Determining a position ofat least one feature of the eye based on at least the reflection of thefirst portion of the image from the eye of the user detected by thefirst narrow waveband photodetector may further include determining aposition of at least one feature of the eye based on the reflection ofthe third portion of the image from the eye of the user detected by thethird narrow waveband photodetector.

The first laser diode may be a red laser diode and outputting a firstvisible laser light in a first narrow waveband by the first laser diodeof the laser module may include outputting a red laser light by the redlaser diode. The first portion of the image may be a red portion of theimage.

The second laser diode may be a green laser diode and outputting asecond visible laser light in a second narrow waveband by the secondlaser diode of the laser module may include outputting a green laserlight by the green laser diode. The second portion of the image may be agreen portion of the image.

The third laser diode may be a blue laser diode and outputting a thirdvisible laser light in a third narrow waveband by the third laser diodeof the laser module may include outputting a blue laser light by theblue laser diode. The third portion of the image may be a blue portionof the image.

A heterogeneous holographic optical element may be summarized asincluding: at least one layer of holographic material, wherein the atleast one layer of holographic material includes: a first hologram toapply a first optical power to light having a first wavelength; and atleast a second hologram to apply at least a second optical power tolight having a second wavelength, the second optical power differentfrom the first optical power and the second wavelength different fromthe first wavelength. The first hologram may redirect light having thefirst wavelength and apply the first optical power to the light havingthe first wavelength upon redirection of the light having the firstwavelength. The second hologram may redirect light having the secondwavelength and apply the second optical power to the light having thesecond wavelength upon redirection of the light having the secondwavelength.

The first optical power may be a positive optical power and the firsthologram may cause light having the first wavelength to converge at afirst rate of convergence. The second optical power may be zero. Thesecond optical power may be a negative optical power and the secondhologram may cause light having the second wavelength to diverge. Thesecond optical power may be positive and less than the first opticalpower, and the second hologram may cause light having the secondwavelength to converge at a second rate of convergence that is less thanthe first rate of convergence. The first optical power may be greaterthan or equal to forty diopters and the second optical power may be lessthan or equal to zero diopters.

The first wavelength may be visible to a human eye and the secondwavelength may be invisible to the human eye. The first wavelength maybe selected from a first range of 390 nm to 700 nm and the secondwavelength may be selected from a second range of 700 nm to 10 um.

The at least one layer of holographic material may include a singlelayer of holographic material and the first hologram and the at least asecond hologram may both be included in the single layer of holographicmaterial. Alternatively, the at least one layer of holographic materialmay include a first layer of holographic material and at least a secondlayer of holographic material, and the first layer of holographicmaterial may include the first hologram and the second layer ofholographic material may include the second hologram.

The at least one layer of holographic material may further include: atleast a third hologram to apply the first optical power to light havinga third wavelength, the third wavelength substantially different fromboth the first wavelength and the second wavelength. The first hologrammay be a red hologram to apply the first optical power to a red light,the second hologram may be an infrared hologram to apply the secondoptical power to an infrared light, the third hologram may be a greenhologram to apply the first optical power to a green light, and the atleast one layer of holographic material may further include a bluehologram to apply the first optical power to a blue light.

The heterogeneous holographic optical element may further include aneyeglass lens, wherein the at least one layer of holographic material iscarried by the eyeglass lens.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

In the drawings, identical reference numbers identify similar elementsor acts. The sizes and relative positions of elements in the drawingsare not necessarily drawn to scale. For example, the shapes of variouselements and angles are not necessarily drawn to scale, and some ofthese elements are arbitrarily enlarged and positioned to improvedrawing legibility. Further, the particular shapes of the elements asdrawn are not necessarily intended to convey any information regardingthe actual shape of the particular elements, and have been solelyselected for ease of recognition in the drawings.

FIG. 1 is an illustrative diagram showing a side view of a wearableheads-up display that employs a scanning laser projector.

FIG. 2 is an illustrative diagram showing a side view of a wearableheads-up display that employs a scanning laser projector and a separateeye tracking system.

FIG. 3 is an illustrative diagram showing a wearable heads-up displaythat includes a scanning laser projector that has been adapted tointegrate eye tracking functionality in accordance with the presentsystems, devices, and methods.

FIG. 4 is an illustrative diagram showing a side view of a wearableheads-up display that is adapted to integrate eye tracking functionalityinto a scanning laser projection system in accordance with the presentsystems, devices, and methods.

FIG. 5 is a perspective view of a wearable heads-up display thatintegrates eye tracking and scanning laser projection with minimalcomponent additions in accordance with the present systems, devices, andmethods.

FIG. 6 is a schematic diagram of an adapted optical splitter forseparating the output of a scanning projector into three angle-separatedcopies in accordance with the present systems, devices, and methods.

FIG. 7 is a flow-diagram showing a method of operating a laser projectorto project an image to an eye of a user and to track the eye of the userin accordance with the present systems, devices, and methods.

FIG. 8 is an illustrative diagram showing a side view of a wearableheads-up display that includes a multiplexed holographic optical elementthat enables both image projection and eye tracking functionality inaccordance with the present systems, devices, and methods.

DETAILED DESCRIPTION

In the following description, certain specific details are set forth inorder to provide a thorough understanding of various disclosedembodiments. However, one skilled in the relevant art will recognizethat embodiments may be practiced without one or more of these specificdetails, or with other methods, components, materials, etc. In otherinstances, well-known structures associated with portable electronicdevices and head-worn devices, have not been shown or described indetail to avoid unnecessarily obscuring descriptions of the embodiments.

Unless the context requires otherwise, throughout the specification andclaims which follow, the word “comprise” and variations thereof, suchas, “comprises” and “comprising” are to be construed in an open,inclusive sense, that is as “including, but not limited to.”

Reference throughout this specification to “one embodiment” or “anembodiment” means that a particular feature, structures, orcharacteristics may be combined in any suitable manner in one or moreembodiments.

As used in this specification and the appended claims, the singularforms “a,” “an,” and “the” include plural referents unless the contentclearly dictates otherwise. It should also be noted that the term “or”is generally employed in its broadest sense, that is as meaning “and/or”unless the content clearly dictates otherwise.

The headings and Abstract of the Disclosure provided herein are forconvenience only and do not interpret the scope or meaning of theembodiments.

The various embodiments described herein provide systems, devices, andmethods for integrating eye tracking functionality into a scanning laserprojector (“SLP”). An aspect hereof includes operating a SLP as both aprojector and as a component of an eye tracker. While applicable in manydifferent use cases, the present systems, devices, and methods areparticularly well-suited for use in wearable heads-up displays (“WHUDs”)that already employ at least one SLP. In accordance with the presentsystems, devices, and methods, a SLP in a WHUD may be adapted tosimultaneously provide visible light for display purposes and infraredlight for eye tracking purposes, thereby enabling eye trackingfunctionality in the WHUD with the addition of only a small number ofdiscreet, unobtrusive components.

The present systems, devices, and methods are well-suited for use inWHUDs, and particularly in WHUDs that already employ at least one SLP.Examples of such displays are described in U.S. Provisional PatentApplication Ser. No. 62/017,089; U.S. Provisional Patent ApplicationSer. No. 62/053,598; U.S. Provisional Patent Application Ser. No.62/117,316; U.S. Provisional Patent Application Ser. No. 62/134,347 (nowU.S. Non-Provisional patent application Ser. No. 15/070,887); U.S.Provisional Patent Application Ser. No. 62/156,736; U.S. ProvisionalPatent Application Ser. No. 62/242,844; US Patent Publication No. US2015-0378164 A1; US Patent Publication No. US 2015-0378161 A1; US PatentPublication No. US 2015-0378162 A1; U.S. Non-Provisional patentapplication Ser. No. 15/145,576; U.S. Non-Provisional patent applicationSer. No. 15/145,609; U.S. Non-Provisional patent application Ser. No.15/145,583; U.S. Non-Provisional patent application Ser. No. 15/046,234;U.S. Non-Provisional patent application Ser. No. 15/046,254; and U.S.Non-Provisional patent application Ser. No. 15/046,269. A generalizedexample of such a WHUD architecture, without eye-tracking capability, isprovided in FIG. 1.

FIG. 1 is an illustrative diagram showing a side view of a WHUD 100 thatemploys a SLP 110. SLP 110 comprises a laser module 111 that includes ared laser diode (labelled “R” in FIG. 1), a green laser diode (labelled“G” in FIG. 1), and a blue laser diode (labelled “B” in FIG. 1), and ascan mirror 112 that is controllably rotatable about two axes offreedom. A single scan mirror 112 that is rotatable about two axes offreedom is used only as an illustrative example herein and a person ofskill in the art will appreciate that similar functionality may berealized using a different mirror configuration, such as for example twoscan mirrors that are each controllably rotatable about a respective oneof two orthogonal axes of freedom and respectively positioned insequence with respect to the optical path of laser light 120. Laserlight 120 output by SLP 110 may comprise any modulated combination ofred laser light (output by the red laser diode), green laser light(output by the green laser diode), and/or blue laser light (output bythe blue laser diode). Laser light 120 reflected from scan mirror 112 isincident on a holographic optical element (“HOE”) 130 that redirectslaser light 120 back towards an eye 190 of a user. Generally, in thepresent systems, devices, and methods, the term “user” refers to a userof a SLP. In the specific context of FIG. 1, the term “user” refers to aperson wearing or using WHUD 100. A person of skill in the art willappreciate that WHUD 100 may include a support frame and/or othersupport/alignment structure(s) (not depicted in FIG. 1 to reduceclutter) that enable a user to wear the elements depicted in FIG. 1 sothat at least HOE 130 is positioned within a field of view of at leastone eye 190 of the user when WHUD 100 is worn on a head of the user.

HOE 130 may be substantially optically transparent to environmentallight 140 (i.e., optically transparent to the majority of wavelengthsthat make up environmental light 140) incident from the opposite side ofHOE 130 relative to laser light 120. Because HOE 130 effectivelycombines projected laser light 120 and external environmental light 140in the user's field of view, HOE 130 may be referred to as a “combiner”or related variant, such as “transparent combiner,” “holographic opticalcombiner,” or similar. If the support frame (not illustrated) of WHUD100 has the general shape, appearance, and/or geometry of a pair ofeyeglasses, then HOE 130 may be carried on one or more transparentlens(es) of WHUD 100 (such as one or more prescription lenses or one ormore non-prescription lenses). Further details on the composition of HOE130 (e.g., including exemplary multiplexed configurations of HOE 130)and on ways in which HOE 130 may redirect laser light 120 towards eye190 (e.g., including exemplary exit pupil and eyebox configurations) aredescribed in at least the patent applications listed above.

WHUD 100 is an example of a WHUD that employs a SLP 110 but does notprovide any eye tracking functionality. An example of how conventionaleye tracking functionality may be added to WHUD 100 is illustrated inFIG. 2.

FIG. 2 is an illustrative diagram showing a side view of a WHUD 200 thatemploys a SLP 210 and a separate eye tracking system. WHUD 200 issubstantially similar to WHUD 100 from FIG. 1, except WHUD 200 includesan eye tracking system comprising additional components 240 and 250 toenable eye tracking functionality in WHUD 200. The eye tracking systemof WHUD 200 includes an infrared light source 240 and an infraredphotodetector 250. In use, infrared light source 240 completelyilluminates or “floods” the eye 290 with a single large spot of infraredlight 222 (drawn in dashed lines to denote that the infrared light 222is invisible to eye 290, and to distinguish from visible light 221output by SLP 210). Infrared photodetector 250 detects reflections ofthe infrared light 222 from the user's eye 290. Different features ofeye 290 (e.g., the cornea, the pupil, the iris, and/or retinal bloodvessels) can cause portions of the single large spot of incidentinfrared light 222 to reflect from eye 290 in different ways; thus, thelocation of such feature(s) of eye 290 relative to infrared light source240 and photodetector 250 can influence the intensity of infrared light222 detected by photodetector 250. As infrared light source 240 floodseye 290 with infrared light, photodetector 250 detects an intensitypattern or map of reflected infrared light 222 that depends on theposition/orientation of eye 290. That is, the intensity of infraredlight 222 detected by photodetector 250 depends on theposition/orientation of eye 290 (or the position/orientation offeature(s) of eye 290, such as the cornea, pupil, and so on). Theintensity pattern/map detected by photodetector 250 depends on where eye290 is looking. In this way, the combination of discrete components(infrared light source 240 and infrared photodetector 250) in the eyetracking system of WHUD 200 enable both the gaze direction and movementsof eye 290 to be measured and tracked.

WHUD 200 depicts an example architecture in which a SLP 210 and an eyetracking system (comprising infrared light source 240 and infraredphotodetector 250) are both included as completely separate andindependent subsystems. Such an implementation may be acceptable forsome systems, but in general it is advantageous for a WHUD to be ascompact and streamlined as possible, both in terms of form factor andprocessing/power requirements. The various embodiments described hereinprovide systems, devices, and methods for integrating eye trackingfunctionality into a SLP to provide a more efficient system in terms ofform factor and processing/power requirements.

FIG. 3 is an illustrative diagram showing a WHUD 300 that includes a SLP310 with an integrated eye tracking functionality in accordance with thepresent systems, devices, and methods. WHUD 300 is substantially similarto WHUD 200 from FIG. 2, except that in WHUD 300 scanning laserprojection and eye tracking components are both integrated into a singlepackage/module 310. Specifically, SLP 310 comprises a laser module 311that includes red laser diode (labelled “R” in FIG. 3), a green laserdiode (labelled “G” in FIG. 3), and a blue laser diode (labelled “B” inFIG. 3) and a scan mirror 312 in a similar configuration to thatdescribed for WHUD 100 of FIG. 1. However, in addition, laser module 311also includes an infrared laser diode (labelled “IR” in FIG. 3) for usein eye tracking in a similar way to that described for infrared lightsource 240 in WHUD 200. Scan mirror 312 simultaneously serves as boththe scan mirror for laser projection (in a similar way to scan mirror112 from WHUD 100 of FIG. 1) and a scan mirror for eye tracking, wherebyscan mirror 312 scans infrared laser light (represented by dashed lines322 in FIG. 3) over the area of eye 390 to sequentially illuminate theentire area of eye 390 (e.g., via a raster scan of IR light). While WHUD200 includes an infrared light source 240 that is separate from theprojector laser module 211, in WHUD 300 infrared laser diode 341 isintegrated into laser module 311 of SLP 310 and scan mirror 312 servesto scan both visible (R, G, and/or B) and infrared (IR) laser light overeye 390.

Scan mirror 312 may advantageously include one or multiple (e.g., in aDLP configuration) digital microelectromechanical systems (“MEMS”)mirror(s). In typical operation, scan mirror 312 of SLP 310 repeatedlyscans over its entire range of positions and effectively scans over theentire field of view of the display. Whether or not an image/pixel isprojected at each scan position depends on controlled modulation oflaser module 311 and its synchronization with scan mirror 312. The factthat scan mirror 312 generally scans over its entire range duringoperation as a laser projector makes scan mirror 312 of SLP 310compatible with use for eye tracking purposes. SLP 310 is adapted toprovide eye tracking functionality without having to compromise ormodify its operation as a SLP. In operation, scan mirror 312 repeatedlyscans over its entire range of positions while the RGB laser diodes aremodulated to provide the visible light 321 corresponding to pixels of ascanned image. At the same time, the infrared laser diode may beactivated to illuminate the user's eye 390 (one spot or pixel at a time,each corresponding to a respective scan mirror position) with infraredlaser light 322 for eye tracking purposes. Depending on theimplementation, the infrared laser diode may simply be on at all timesto completely illuminate (i.e., scan over the entire area of) eye 390with infrared laser light 322 or the infrared laser diode may bemodulated to provide an illumination pattern (e.g., a grid, a set ofparallel lines, a crosshair, or any other shape/pattern) on eye 390.Because infrared laser light 322 is invisible to eye 390 of the user,infrared laser light 322 does not interfere with the scanned image beingprojected by SLP 310.

In order to detect the (e.g., portions of) infrared laser light 322 thatreflects from eye 390, WHUD 300 includes at least one infraredphotodetector 350 similar to photodetector 250 from WHUD 200 of FIG. 2.While only one photodetector 350 is depicted in FIG. 3, in alternativeembodiments any number of photodetectors 350 may be used (e.g., an arrayof photodetectors 350, or a charge-coupled device based camera that isresponsive to light in the infrared wavelength range) positioned in anyarrangements and at any desired location(s) depending on theimplementation.

As scan mirror 312 scans modulated R, G, and/or B light 321 over eye 390to produce a displayed image based on modulation of the R, G, and/or Blaser diodes, scan mirror 312 also scans infrared laser light 322 overeye 390 based on modulation of the IR laser diode. Photodetector 350detects an intensity pattern or map of reflected infrared laser light322 that depends on the position/orientation of eye 390. That is, eachdistinct position of scan mirror 312 may result in a respectiveintensity of infrared laser light 322 being detected by photodetector350 that depends on the position/orientation of eye 390 (or theposition/orientation of feature(s) of eye 390, such as the cornea, iris,pupil, and so on). The intensity pattern/map detected by photodetector350 depends on where eye 390 is looking. In this way, the same SLP 310in WHUD 300 enables both i) image projection, and ii) the gaze directionand movements of eye 390 to be measured and tracked.

Another adaptation to WHUD 300 relative to WHUD 200, for the purpose ofintegrating eye tracking functionality into SLP 310, iswavelength-multiplexing of HOE 330. In the same way as described for HOE130 of WHUD 100, WHUD 300 also includes a HOE 330 that redirects laserlight output from the laser module 311 of SLP 310 towards eye 390;however, in WHUD 300, HOE 330 has been adapted (relative to HOE 130 ofFIG. 1) to include at least two wavelength-multiplexed holograms: atleast a first hologram 331 that is responsive to (i.e., redirects atleast a portion of, the magnitude of the portion depending on theplayback efficiency of the first hologram) the visible light 321 outputby laser module 311 and unresponsive to (i.e., transmits) the infraredlight 322 output by laser module 311, and a second hologram 332 that isresponsive to (i.e., redirects at least a portion of, the magnitude ofthe portion depending on the playback efficiency of the second hologram)the infrared light 322 output by laser module 311 and unresponsive to(i.e., transmits) the visible light 321 output by laser module 311.While FIG. 3 depicts first hologram 331 as a single hologram, inpractice the aspect(s) of HOE 330 that is/are responsive to the visiblelight 321 output by laser module 311 may include any number of hologramsthat may be multiplexed in a variety of different ways, includingwithout limitation: wavelength multiplexed (i.e., a “red” hologram thatis responsive to only red light from the red laser diode of laser module311, a “green” hologram that is responsive to only green light from thegreen laser diode of laser module 311, and a “blue” hologram that isresponsive to only blue light from the blue laser diode of laser module311), angle multiplexed (e.g., for the purpose of eye boxexpansion/replication), phase multiplexed, spatially multiplexed, and soon. Upon redirection of visible light 321, first hologram 331 may applya first optical power to visible light 321. Advantageously, the firstoptical power applied by first hologram 331 (or by the first set ofmultiplexed holograms if the implementation employs a set of multiplexedholograms for redirecting the visible light 321) may be a positiveoptical power that focuses or converges the visible light 321 to, forexample, an exit pupil having a diameter less than one centimeter (e.g.,6 mm, 5 mm, 4 mm, 3 mm) at the eye 390 of the user for the purpose ofproviding a clear and focused image with a wide field of view. Uponredirection of infrared light 322, second hologram 332 may apply asecond optical power to infrared light 322, where the second opticalpower applied by second hologram 332 is different from the first opticalpower applied by first hologram 331. Advantageously, the first opticalpower may be greater than the second optical power (and therefore, thesecond optical power may be less than the first optical power) so thatsecond hologram 332 redirects infrared light 322 over an area of eye 390that is larger than the exit pupil of visible light 321 at eye 390. Forexample, the second optical power of second hologram 332 may apply arate of convergence to infrared light 322 that is less than the rate ofconvergence applied to visible light 321 by the first optical power offirst hologram 331, or the second optical power may be zero such thatsecond hologram 332 redirects infrared light 322 towards eye 390 withoutapplying any convergence thereto, or the second optical power may benegative (i.e., less than zero) so that the second optical power ofsecond hologram 332 causes infrared light 322 to diverge (i.e., appliesa rate of divergence thereto) to cover, for example, cover the entirearea of eye 390 (and beyond, if desired) for the purpose of illuminatingthe entire area of eye 390 and tracking all eye positions/motions withinthat illuminated area.

Depending on the specific implementation, HOE 330 may comprise a singlevolume of holographic material (e.g., photopolymer or a silver halidecompound) that encodes, carries, has embedded therein or thereon, orgenerally includes both first hologram 331 and second hologram 332, oralternatively HOE 330 may comprise at least two distinct layers ofholographic material (e.g., photopolymer and/or a silver halidecompound) that are laminated or generally layered together, a firstlayer of holographic material that includes first hologram 331 and asecond layer of holographic material that includes second hologram 332.More details of an exemplary multiplexed HOE are described later onewith reference to FIG. 8.

Throughout this specification and the appended claims, the term“infrared” includes “near infrared” and generally refers to a wavelengthof light that is larger than the largest wavelength of light that istypically visible to the average human eye. Light that is visible to theaverage human eye (i.e., “visible light” herein) is generally in therange of 400 nm-700 nm, so as used herein the term “infrared” refers toa wavelength that is greater than 700 nm, up to 1 mm.

The use of infrared light is advantageous in eye tracking systemsbecause infrared light is invisible to the (average) human eye and sodoes not disrupt or interfere with other optical content being displayedto the user. Integrating an infrared laser diode into a SLP, inaccordance with the present systems, devices, and methods, enablesvisible laser projection and invisible eye tracking to be simultaneouslyperformed by substantially the same hardware of a WHUD, therebyminimizing overall bulk and processing/power requirements of the system.However, the various embodiments described herein also include systems,devices, and methods of integrating eye tracking functionality into aSLP operated completely in the visible spectrum (i.e., without infraredlight).

FIG. 4 is an illustrative diagram showing a side view of a WHUD 400 thatis adapted to integrate eye tracking functionality into a scanning laserprojection system in accordance with the present systems, devices, andmethods. WHUD 400 is substantially similar to WHUD 100 from FIG. 1,except that WHUD 400 includes at least three narrow wavebandphotodetectors 451, 452, and 453 to detect visible laser light 420 (asopposed to at least one infrared photodetector 350 to detect infraredlaser light) reflected from an eye 490 of a user and to use theresulting intensity pattern/map to determine the position and/ormovements of eye 490.

WHUD 400 comprises a SLP 410 that includes three narrow waveband lightsources: a red laser diode (labelled “R” in FIG. 4), a green laser diode(labelled “G” in FIG. 4), and a blue laser diode (labelled “B” in FIG.4). Throughout this specification and the appended claims, the term“narrow waveband” refers to a relatively small range of wavelengths (orwavelength bandwidth) given the specific context. In the context of alight source such as a laser diode, a narrow waveband light is lightwithin a bandwidth of about 10 nm or less; in the context of aphotodetector, a narrow waveband photodetector is responsive to lightwithin a bandwidth of about 200 nm or less. Laser light 420 from SLP 410is modulated to project an image on the eye 490 of the user as describedfor WHUD 100 of FIG. 1. However, WHUD 400 also includes: a first narrowwaveband photodetector 451 responsive to laser light 420 in the narrowwaveband corresponding to light output by the red laser diode of SLP410, a second narrow waveband photodetector 452 responsive to laserlight 420 in the narrow waveband corresponding to light output by thegreen laser diode of SLP 410, and a third narrow waveband photodetector453 responsive to laser light 420 in the narrow waveband correspondingto light output by the blue laser diode of SLP 410. Each ofphotodetectors 451, 452, and 453 is aligned to receive laser light 420reflected from the eye 490 of the user to enable the position and/ormotion of eye 490 to be determined. Each photodetector may be adapted tobe responsive to a respective “narrow waveband” of light using one moreoptical filters, such as one or more optical bandpass filters.Photodetectors 451, 452, and 453 are advantageously “narrow waveband” tominimize noise from detected environmental light.

WHUD 400 implements laser eye tracking using the same visible laserlight 420 that also corresponds to images/pixels projected on the eye490 of the user from SLP 410. An advantage to this scheme is that noinfrared laser diode is required and SLP 410 may be used essentiallywithout modification; however, a disadvantage is that the eyepositions/motions must be determined subject to light from a projectedimage/pixel pattern instead of using the full invisible illuminationafforded by infrared light. In accordance with the present systems,devices, and methods, communication between the image generation systemof a SLP (i.e., the system that controls the modulation of laser light420 in synchronization with the positions of the scan mirror) and theeye tracking system that determines the position/motion of eye 490 basedon reflected light detected by narrow waveband photodetectors 451, 452,and 453 is advantageous. Such communication may include, for example,information about which laser diode is active at each mirror position.Using this information, the eye tracking system is able to map detectedintensity information from photodetector(s) 451, 452, and/or 453 tovarious positions and/or motions of eye 490 based on the current scanmirror position and laser modulation pattern.

The various embodiments of eye tracking systems and devices describedherein may, in some implementations, make use of “glint” and/or“Purkinje images” and/or may employ the “corneal shadow based” methodsof eye tracking described in U.S. Provisional Patent Application Ser.No. 62/245,792.

In accordance with the present systems, devices, and methods, an eyetracking system (or an “eye tracker”) may include one or more digitalprocessor(s) communicatively coupled to the one or more (narrowwaveband) photodetector(s) and to one or more non-transitoryprocessor-readable storage medium(ia) or memory(ies). The memory(ies)may store processor-executable instructions and/or data that, whenexecuted by the processor, enable the processor to determine theposition and/or motion of an eye of the user based on information (e.g.,intensity information, such as an intensity pattern/map) provided by theone or more photodetector(s).

FIG. 5 is a perspective view of a WHUD 500 that integrates eye trackingand scanning laser projection with minimal component additions inaccordance with the present systems, devices, and methods. WHUD 500includes many of the elements depicted in FIGS. 1, 2, 3, and 4, namely:a laser module 511 adapted to output a first visible laser light 521(e.g., in at least a first narrow waveband) representative of a firstportion of an image (e.g., a red portion of the image) and a secondvisible laser light 522 in a second narrow waveband representative of asecond portion of the same image (e.g., a green portion of the image), ascan mirror aligned to receive visible laser light output by the lasermodule and controllably reflect (i.e., scan) the visible laser light, awavelength-multiplexed HOE 530 aligned to redirect the visible laserlight 521 and 522 (i.e., the first portion of the image 521 and thesecond portion of the image 522) towards an eye 590 of a user, and arespective narrow waveband photodetector 550 for each respective narrowwaveband laser diode (only one photodetector illustrated in FIG. 5),each narrow waveband photodetector 550 responsive to light in arespective narrow waveband corresponding to a respective one of thenarrow waveband laser diodes. As used herein and in the claims, visiblemeans that the light includes wavelengths within the human visibleportion of the electromagnetic spectrum, typically from approximately400 nm (violet) to approximately 700 nm (red). Depending on theimplementation, the first visible laser light 521 (i.e., the firstportion of the image) in the first narrow waveband may correspond to anyvisible light such as red, green, or blue, and the second visible laserlight 522 in the second narrow waveband (i.e., the second portion of theimage) may correspond to any other visible light (e.g., if the firstvisible laser light is red, the second visible laser light may be greenor blue, and so on). WHUD 500 also includes a support frame 580 that hasa general shape and appearance or a pair of eyeglasses, so that HOE 530is positioned within a field of view of the eye 590 of the user whensupport frame 580 is worn on a head of the user.

WHUD 500 further includes a digital processor 560 communicativelycoupled to photodetector 550 and a non-transitory processor-readablestorage medium or memory 570 communicatively coupled to digitalprocessor 570. Memory 570 stores processor-executable instructionsand/or data that, when executed by processor 560, cause processor 560 todetermine one or more position(s) and/or movement(s) of eye 590 based oninformation about visible light 521 (i.e., the first portion of theimage) and/or visible light 522 (i.e., the second portion of the image)reflected from eye 590 communicated to processor 560 fromphotodetector(s) 550.

The various embodiments described herein generally reference andillustrate a single eye of a user (i.e., monocular applications), but aperson of skill in the art will readily appreciate that the presentsystems, devices, and methods may be duplicated in a WHUD in order toprovide scanned laser projection and scanned laser eye tracking for botheyes of the user (i.e., binocular applications).

Some WHUDs (e.g., those that implement certain eyeboxreplication/expansion schemes) may involve various optical elements inthe path of the laser light output by the SLP. In accordance with thepresent systems, devices, and methods, WHUDs that integrate an infraredlaser diode into the SLP for eye tracking purposes may advantageouslyemploy hot optical elements and/or cold optical elements as needed inorder to align/separate the respective paths of the visible and infraredlasers. An example is depicted in FIG. 6.

FIG. 6 is a schematic diagram of an adapted optical splitter 600 forseparating the output of a SLP into three angle-separated copies asdescribed in U.S. Provisional Patent Application Ser. No. 62/156,736 andU.S. Provisional Patent Application Ser. No. 62/242,844 (now U.S.Non-Provisional patent application Ser. No. 15/046,254). Splitter 600includes an optical structure 670 having two reflective surfaces 671 and672 oriented at respectively different angles and a transmissive region673 therebetween. A SLP 610 (which may be substantially similar to SLP310 from FIG. 3) has a scan range that includes subranges A, B, and C asindicated in FIG. 6. SLP 610 may be operated to scan visible light overthree copies of an image: a first copy in scan range A, a second copy inscan range B, and a third copy in scan range C. The first copy of theimage projected over scan range A is transmitted through transmissiveregion 673 of optical structure 670 to impinge on, for example, anangle-multiplexed holographic combiner. A second copy of the imageprojected over scan range B is reflected by first reflective surface 671of optical structure 670 and then reflected again by a second reflector(e.g., mirror) 681. Second reflector 681 is oriented to redirect lightcorresponding to scan range B towards the holographic combiner (notshown in FIG. 6 to reduce clutter). A third copy of the image projectedover scan range C is reflected by second reflected surface 672 ofoptical structure 670 and then reflected again by a third reflector(e.g., mirror) 682. Third reflector 682 is oriented to redirect lightcorresponding to scan range C towards the holographic combiner. The samemodulation pattern of laser light may be repeated by SLP 610 over eachof ranges A, B, and C and, in this way, three copies of an image may beproduced by SLP 610 and directed towards an angle-multiplexedholographic combiner at respectively different angles. Optical splitter600 represents an example of a configuration of an optical splitter thatmay be used in conjunction with an accordingly adapted SLP operationalmode and an angle-multiplexed holographic combiner in order to expandthe eyebox of a retinal scanning display system by exit pupilreplication. In order to integrate infrared laser light, for eyetracking purposes, into a system that employs such a splitter, thesplitter may, for example, be constructed of cold optical elements suchthat the infrared light is transmitted therethrough essentially without“seeing” or being influenced by the splitter. In this case, the infraredlight (represented by dashed lines in FIG. 6) may be scanned over theentire range of A+B+C. Alternatively, the splitter 600 may beconstructed of hot optical elements such that the infrared light isreflected thereby. In this case, the infrared light may only need to bemodulated on for one of the three scan regions A, B, or C and modulatedoff for the other two scan regions.

FIG. 7 is a flow-diagram showing a method 700 of operating a laserprojector to project an image to an eye of a user and to track the eyeof the user in accordance with the present systems, devices, andmethods. Method 700 includes five acts 701, 702, 703, 704, and 705,though those of skill in the art will appreciate that in alternativeembodiments certain acts may be omitted and/or additional acts may beadded. Those of skill in the art will also appreciate that theillustrated order of the acts is shown for exemplary purposes only andmay change in alternative embodiments. For the purpose of method 700,the term “user” refers to a person that is observing an image projectedby the laser projector.

At 701, a laser module of the laser projector outputs visible laserlight that is representative of at least a portion of an image. Thelaser module includes at least a first laser diode and at 701 the firstlaser diode outputs a first visible laser light in a first narrowwaveband. The first visible laser light output by the first laser diodeis representative of a first portion of the image. For example, thefirst laser diode may be a laser diode of a particular color (e.g., ared laser diode, a green laser diode, or a blue laser diode) and thefirst visible laser light in the first narrow waveband may represent acorresponding color portion of the image (e.g., a red portion of theimage if the first laser diode is a red laser diode, a green portion ofthe image if the first laser diode is a green laser diode, or a blueportion of the image if the first laser diode is a blue laser diode).

At 702, a scan mirror in the laser projector controllably andreflectively scans the visible laser light.

At 703, the laser light reflected from the scan mirror is received by aHOE and redirected thereby or therefrom towards an eye of a user.

At 704, at least a first narrow waveband photodetector detects areflection of at least a portion the visible laser light from the eye ofthe user. The intensity of the visible laser light detected by thephotodetector may depend on the position, orientation, and/or movementof one or more feature(s) of the eye from which the visible laser lightreflected. The first narrow waveband photodetector may be responsive tolight in the first narrow waveband (e.g., the first visible laser lightoutput by the first laser diode at 701) and substantially unresponsiveto light that is outside of the first narrow waveband. Generally,detecting a reflection of the at least a portion of the visible laserlight from the eye of the user by at least a first narrow wavebandphotodetector at 704 includes detecting a reflection of the firstportion of the image by the first narrow waveband photodetector. Inother words, at 704 the first narrow waveband photodetector detects areflection of the first portion of the projected image (e.g., a redportion of the image if the first laser diode is a red laser diode andthe first narrow waveband photodetector is a red photodetector) from theeye of the user.

At 705, a position of at least one feature of the eye of the user isdetermined based on at least the reflection of the first portion of theimage detected by the first narrow waveband photodetector at 704. The atleast one feature of the eye may include, without limitation, a pupil,iris, cornea, or retinal blood vessel of the eye. In this context, theterm “position” is used loosely to refer to the general spatial locationand/or orientation of the at least one feature of the eye with respectto a reference point, such as the spatial location and/or orientation ofthe photodetector or a previously known spatial location and/ororientation of the at least one feature. Accordingly, the position ofthe at least one feature of the eye determined at 705 may berepresentative of (and/or used to subsequently determine) the position,orientation, and/or motion of the eye itself. In some implementations,the position of the at least one feature of the eye (and/or thecorresponding position, orientation, and/or motion of the eye itself)may be determined by a processor in communication with at least thefirst narrow waveband photodetector.

In some implementations, the laser module may include a second laserdiode and at 701 the second laser diode outputs a second visible laserlight in a second narrow waveband that is different from the firstnarrow waveband. The second visible laser light output by the secondlaser diode is representative of a second portion of the image (e.g., ifthe first laser diode is a red laser diode and the first portion of theimage is a red portion of the image, then the second laser diode may bea green laser diode and the second portion of the image may be a greenportion of the image). In such implementations, the scan mirror maycontrollably and reflectively scan both the first portion of the imageand the second portion of the image at 702 and the HOE may be awavelength-multiplexed HOE that redirects both the first portion of theimage and the second portion of the image towards the eye of the user at703. The wavelength-multiplexed HOE may include a first hologram that isresponsive to light in the first narrow waveband (e.g., the firstportion of the image) and unresponsive to light that is outside thefirst narrow waveband and a second hologram that is responsive to lightin the second narrow waveband (e.g., the second portion of the image)and unresponsive to light that is outside the second narrow waveband,and at 703 the first hologram may redirect the first portion of theimage towards the eye of the user and the second hologram may redirectthe second portion of the image towards the eye of the user.

Implementations that employ multiple narrow waveband laser diodes mayalso employ multiple narrow waveband photodetector, with a respectivenarrow waveband photodetector wavelength-matched to each respectivenarrow waveband laser diode. For example, implementations that employ ared laser diode, a green laser diode, and a blue laser diode may alsoemploy a red photodetector, a green photodetector, and a bluephotodetector. In this case, detecting a reflection of at least aportion of the visible laser light from the eye of the user by at leasta first narrow waveband photodetector at 704 may include any or all of:detecting a reflection of a first portion (e.g., a red portion) of aprojected image from the eye of the user by a first narrow wavebandphotodetector (e.g., a red photodetector), detecting a reflection of asecond portion (e.g., a green portion) of the image from the eye of theuser by a second narrow waveband photodetector (e.g., a greenphotodetector), and/or detecting a reflection of a third portion (e.g.,a blue portion) of the image from the eye of the user by a third narrowwaveband photodetector (e.g., a blue photodetector). Each narrowwaveband photodetector may be responsive to light in a respective narrowwaveband and substantially unresponsive to light that is outside of thatnarrow waveband. It follows, then, in accordance with the presentsystems, devices, and methods, that determining a position of at leastone feature of the eye based on at least the reflection of the firstportion of the image from the eye of the user detected by the firstnarrow waveband photodetector at 705 may include any or all of:determining a position of at least one feature of the eye based on thereflection of the first portion of the image (e.g., the red portion ofthe image) from the eye of the user detected by the first narrowwaveband photodetector (e.g., the red photodetector), determining aposition of at least one feature of the eye based on the reflection ofthe second portion of the image (e.g., the green portion of the image)from the eye of the user detected by the second narrow wavebandphotodetector (e.g., the green photodetector), and/or determining aposition of at least one feature of the eye based on the reflection ofthe third portion of the image (e.g., the blue portion of the image)from the eye of the user detected by the third narrow wavebandphotodetector (e.g., the blue photodetector).

In some implementations, multiple photodetectors may be used to detectreflections of narrow waveband light from the eye of the user, and themultiple photodetectors may be physically clustered together orspatially separated around the support frame of a WHUD (e.g., around aperimeter of the HOE).

Where infrared light is used to illuminate all or a portion of the eyefor eye tracking purposes, the full area of the eye may be completelyilluminated via a full raster scan, or (since the projector isrefreshing each frame quickly and full eye tracking can be spread outover multiple frames without noticeable delay to the user) portions ofthe eye may be illuminated in any of various patterns. For example,passive patterns such as a grid or set of parallel lines may beemployed, or active patterns may be employed. Examples of activeillumination patterns include: “binary style search” in which the areaof the eye is divided into binary regions, the eye tracker determineswhich of the two regions contains a feature (e.g., the pupil or cornea),that region is subsequently divided into binary regions, and the processis continued with smaller and smaller regions until the position of thefeature is identified with the desired resolution; “recent area focus”in which once a trusted eye position is found subsequent scans arelimited to a subset of the full scan area that includes the position ofthe known eye position, with the subset being based on the likelihood ofwhere the eye could possibly move within the time since the trusted eyeposition was identified; and/or “rotary scan” in which the area of theeye is divided into wedges or pie pieces which are scanned insuccession.

The use of infrared light is advantageous because such light is readilydistinguishable from the visible light provided by the laser projector.However, infrared light is also prevalent in the environment so a narrowwaveband photodetector that is optimized to be responsive to infraredlight may nevertheless detect environmental infrared noise. In order tohelp mitigate this effect (both in the infrared regime and inimplementations in which visible light is used for eye tracking, e.g.,as depicted in FIG. 4), laser light that is used for eye trackingpurposes may be encoded in any of a variety of different ways to enablesuch light to be distinguished from environmental light of a similarwavelength. For example, narrow waveband light (infrared or visible)that is used for eye tracking purposes may be deliberately polarized anda corresponding polarization filter may be applied to the narrowwaveband (e.g., infrared) photodetector so that the photodetector isonly responsive to light that is in the narrow waveband and of thecorrect polarization. As another example, narrow waveband light that isused for eye tracking purposes may be modulated with a deliberatemodulation pattern and light providing this pattern can be extractedfrom the intensity map provided by the photodetector during the signalprocessing and analysis of the photodetector output. In someimplementations, an infrared filter may be applied to or otherwiseintegrated with the lens (transparent combiner) of a WHUD to blockinfrared light from the user's external environment from passing throughthe lens/transparent combiner and impinging on the eye of the user, sothat the amount of environmental infrared light that is reflected fromthe eye and detected by an infrared photodetector is reduced.

As described previously, integrating infrared laser light into the SLPof a WHUD for eye tracking purposes may advantageously employ a HOE thatis designed to impart a different optical function (e.g., optical power)on infrared laser light from the optical function that it imparts on thevisible laser light.

FIG. 8 is an illustrative diagram showing a side view of a WHUD 800 thatincludes a wavelength-multiplexed HOE 830 that enables both imageprojection and eye tracking functionality in accordance with the presentsystems, devices, and methods. WHUD 800 is substantially similar to WHUD300 from FIG. 3 with some details of HOE 830 enhanced for the purpose ofillustration. In brief, WHUD 800 includes a SLP 810 adapted to includean infrared laser diode (labeled as “IR” in FIG. 8) for eye trackingpurposes and a transparent combiner comprising a wavelength-multiplexedHOE 830 integrated with (e.g., laminated or otherwise layered upon, orcast within) an eyeglass lens 860. Integration of HOE 830 with lens 860may include and/or employ the systems, devices, and methods described inU.S. Provisional Patent Application Ser. No. 62/214,600 and/or U.S.Provisional Patent Application Ser. No. 62/268,892.

HOE 830 is wavelength-multiplexed to respond differently (i.e., apply adifferent optical power to) different wavelengths of light incidentthereon. More specifically, HOE 830 is a heterogeneous HOE including atleast a first hologram that applies a first optical power to light 821having a first wavelength (e.g., at least a first visible wavelength)and a second hologram that applies a second optical power to light 822having a second wavelength (e.g., an infrared wavelength). The secondoptical power is different from the first optical power and the secondwavelength is different from the first wavelength. HOE 830 may includeany number of layers of holographic material (e.g., photopolymer, asilver halide compound) carrying, encoding, containing, or otherwiseincluding any number of holograms. A single layer of holographicmaterial may include multiple holograms and/or individual holograms maybe included on or in respective individual layers of holographicmaterial.

In the illustrated example in FIG. 8, the “light having a firstwavelength” and the “light having a second wavelength” respectivelycorrespond to visible laser light 821 and infrared laser light 822, bothoutput by SLP 810. SLP 810 outputs visible laser light 821 (representedby solid lines in FIG. 8) for the purpose of image projection andinfrared laser light 822 (represented by dashed lines in FIG. 8) for thepurpose of eye tracking. As examples, the visible laser light 821 mayinclude light having at least one wavelength (e.g., red, green, orbelow; or any combination of red, green, and/or blue) in the range ofabout 390 nm to about 700 nm and the infrared laser light 822 mayinclude light having at least one wavelength in the range of about 700nm to about 10 um. Both visible laser light 821 and infrared laser light822 are incident on wavelength-multiplexed HOE 830 and redirectedthereby towards the eye 890 of a user of WHUD 800; however, because therequirements of image projection and eye tracking are different,wavelength-multiplexed HOE 830 redirects visible laser light 821 towardseye 890 in a different way from how wavelength-multiplexed HOE 830redirects infrared laser light 822 towards eye 890.Wavelength-multiplexed HOE 830 includes i) at least a first hologramthat is responsive to (i.e., redirects and applies a first optical powerto) visible laser light 821 (i.e., light having at least a firstwavelength in the visible spectrum) towards eye 890 and, and ii) asecond hologram that is responsive to (i.e., redirects and applies asecond optical power) infrared laser light 822 (i.e., light having asecond wavelength in the infrared spectrum) towards eye 890. The firstoptical power (i.e., the optical power applied to the visible laserlight 821 by at least a first hologram of wavelength-multiplexed HOE830) is positive so that the at least a first hologram inwavelength-multiplexed HOE 830 causes the visible laser light 821 toconverge to a first exit pupil at or near the eye 890 of the user. Thisconvergence is advantageous to enable the user to see displayed contentwith a reasonable field of view. Because wavelength-multiplexed HOE 830is integrated with lens 860, wavelength-multiplexed HOE 830 may bepositioned proximate eye 890 and the first optical power may berelatively high (e.g., greater than or equal to about 40 diopters) inorder to provide the necessary convergence. Concurrently, the secondoptical power (i.e., the optical power applied to the infrared laserlight 822 by a second hologram of wavelength-multiplexed HOE 830) isless than the first optical power applied to the visible light by the atleast a first hologram of wavelength-multiplexed HOE 830. The secondoptical power applied by the second hologram of wavelength-multiplexedHOE 830 may be positive and less than the first optical power applied bythe at least a first hologram of wavelength-multiplexed HOE 830 (e.g.,less than about 40 diopters; enough to reduce a divergence of,collimate, or converge) such that the infrared light 822 converges to anexit pupil that has a larger diameter at eye 890 than the exit pupil ofthe visible light 821. Alternatively, the second optical power appliedby the second hologram may be zero or negative so that the secondhologram of wavelength-multiplexed HOE 830 causes the infrared laserlight 822 to redirect towards 890 without convergence (i.e., as from aplane mirror) or to diverge. In other words, the second optical powermay be less than or equal to about 0 diopters. Providing a larger exitpupil for the infrared light 822 than the visible light 821 at eye 890is advantageous to enable SLP 810 to illuminate the entire area of eye890 with infrared laser light 822 for eye tracking purposes.

In accordance with the present systems, devices, and methods, the atleast a first hologram in wavelength-multiplexed HOE 830 that isresponsive to visible light may include any number ofwavelength-multiplexed holograms, each of which may be responsive to arespective wavelength or respective range of wavelengths of visiblelight. For example, the at least a first hologram inwavelength-multiplexed HOE 830 that is responsive to visible light mayinclude a red hologram that is responsive to red light provided by SLP810, a green hologram that is responsive to green light provided by SLP810, and/or a blue hologram that is responsive to blue light provided bySLP 810. Advantageously, each hologram that is responsive to visiblelight included in the at least a first hologram ofwavelength-multiplexed HOE 830 may apply that same first optical powerto the particular visible light to which the hologram is responsive.

The integration of eye tracking functionality in a WHUD that alreadyemploys a SLP and a holographic combiner for display purposes may, inaccordance with the present systems, devices, and methods, be achievedby mostly discreetly adapting existing hardware components as opposed toadding the bulk of many new components. Specifically, i) an infraredlaser diode may be to the SLP (the infrared diode modulatedindependently of the visible light diode(s) in the projector), ii) aninfrared hologram may be added to the holographic combiner (the infraredhologram applying a lower optical power (including zero or negativeoptical power) to the infrared laser light in order to cover the entireeye area, in contrast to the relatively large optical power applied bythe holographic combiner to the visible laser light), and iii) at leastone infrared photodetector may be added to the WHUD to monitorreflections of the infrared laser light from the eye of the user.

As described previously, both the first hologram and the second hologramof wavelength-multiplexed HOE 830 may be included in or on a singlelayer of holographic material (e.g., film) or, alternatively, the firsthologram may be included in or on a first layer of holographic materialand the second hologram may be included in or on a second layer ofholographic material. In the latter case, the first layer of holographicmaterial and the second layer of holographic material may be laminatedor otherwise layered together either directly or through any number ofintervening layers/materials.

In some implementations, wavelength-multiplexed HOE 830 may include anynumber of additional holograms distributed over any number of layers.For example, wavelength-multiplexed HOE 830 may include a first hologramthat is responsive to a red component of visible laser light 821, asecond hologram that is responsive to infrared laser light 822, a thirdhologram that is responsive to a green component of visible laser light821, and a fourth hologram that is responsive to a blue component ofvisible laser light 821. In this configuration, the first, third, andfourth holograms may each apply a same first optical power to therespective visible light to which each hologram is responsive and thesecond hologram may apply a second optical power to the infrared light.

The various embodiments described herein may be used for other sensingapplications beyond eye tracking. For example, the high resolution andhigh sensitivity eye tracking enabled herein may be processed to extractsubtle causes of eye movements, such as eye saccades and/or a user'sheartbeat and/or a user's blood pressure.

One consequence to integrating eye tracking into a SLP is that theresulting eye tracking capability is only active when the SLP itself isactive. In some situations, it may be desirable to provide a coarse eyetracking functionality even when the SLP is turned off. To this end, thevarious embodiments described herein (e.g., the configurations depictedin FIGS. 3, 4 and 5) may optionally include a separate eye trackingsystem (such as that depicted in FIG. 2) to enable the user to activatethe SLP by glancing at one or more specific location(s). An example of asuitable coarse, supplemental, or second eye tracking system that may becombined in a WHUD employing the SLP-based eye tracking of the presentsystems, devices, and methods is described in U.S. Provisional PatentApplication Ser. No. 62/281,041.

Throughout this specification and the appended claims, reference isoften made to a “laser module,” such as a laser projector (SLP orotherwise) comprising a laser module. Unless the specific contextrequires otherwise, the term “a laser module” should be interpretedloosely to mean “at least one laser module” and the variousimplementations described and claimed herein are generic to the numberof distinct laser modules employed. For example, an SLP may employ asingle laser module that includes any number of laser diodes, or a SLPmay employ multiple laser modules (or a laser equivalent of a multi-chipmodule, such as a multi-chip laser module) that each include any numberof laser diodes.

Throughout this specification and the appended claims, the term “about”is sometimes used in relation to specific values or quantities. Forexample, “light within a bandwidth of about 10 nm or less.” Unless thespecific context requires otherwise, the term about generally means±15%.

The WHUDs described herein may include one or more sensor(s) (e.g.,microphone, camera, thermometer, compass, and/or others) for collectingdata from the user's environment. For example, one or more camera(s) maybe used to provide feedback to the processor of the wearable heads-updisplay and influence where on the transparent display(s) any givenimage should be displayed.

The WHUDs described herein may include one or more on-board powersources (e.g., one or more battery(ies)), a wireless transceiver forsending/receiving wireless communications, and/or a tethered connectorport for coupling to a computer and/or charging the one or more on-boardpower source(s).

Throughout this specification and the appended claims the term“communicative” as in “communicative pathway,” “communicative coupling,”and in variants such as “communicatively coupled,” is generally used torefer to any engineered arrangement for transferring and/or exchanginginformation. Exemplary communicative pathways include, but are notlimited to, electrically conductive pathways (e.g., electricallyconductive wires, electrically conductive traces), magnetic pathways(e.g., magnetic media), and/or optical pathways (e.g., optical fiber),and exemplary communicative couplings include, but are not limited to,electrical couplings, magnetic couplings, and/or optical couplings.

Throughout this specification and the appended claims, infinitive verbforms are often used. Examples include, without limitation: “to detect,”“to provide,” “to transmit,” “to communicate,” “to process,” “to route,”and the like. Unless the specific context requires otherwise, suchinfinitive verb forms are used in an open, inclusive sense, that is as“to, at least, detect,” to, at least, provide,” “to, at least,transmit,” and so on.

The above description of illustrated embodiments, including what isdescribed in the Abstract, is not intended to be exhaustive or to limitthe embodiments to the precise forms disclosed. Although specificembodiments of and examples are described herein for illustrativepurposes, various equivalent modifications can be made without departingfrom the spirit and scope of the disclosure, as will be recognized bythose skilled in the relevant art. The teachings provided herein of thevarious embodiments can be applied to other portable and/or wearableelectronic devices, not necessarily the exemplary wearable electronicdevices generally described above.

For instance, the foregoing detailed description has set forth variousembodiments of the devices and/or processes via the use of blockdiagrams, schematics, and examples. Insofar as such block diagrams,schematics, and examples contain one or more functions and/oroperations, it will be understood by those skilled in the art that eachfunction and/or operation within such block diagrams, flowcharts, orexamples can be implemented, individually and/or collectively, by a widerange of hardware, software, firmware, or virtually any combinationthereof. In one embodiment, the present subject matter may beimplemented via Application Specific Integrated Circuits (ASICs).However, those skilled in the art will recognize that the embodimentsdisclosed herein, in whole or in part, can be equivalently implementedin standard integrated circuits, as one or more computer programsexecuted by one or more computers (e.g., as one or more programs runningon one or more computer systems), as one or more programs executed by onone or more controllers (e.g., microcontrollers) as one or more programsexecuted by one or more processors (e.g., microprocessors, centralprocessing units, graphical processing units), as firmware, or asvirtually any combination thereof, and that designing the circuitryand/or writing the code for the software and or firmware would be wellwithin the skill of one of ordinary skill in the art in light of theteachings of this disclosure.

When logic is implemented as software and stored in memory, logic orinformation can be stored on any processor-readable medium for use by orin connection with any processor-related system or method. In thecontext of this disclosure, a memory is a processor-readable medium thatis an electronic, magnetic, optical, or other physical device or meansthat contains or stores a computer and/or processor program. Logicand/or the information can be embodied in any processor-readable mediumfor use by or in connection with an instruction execution system,apparatus, or device, such as a computer-based system,processor-containing system, or other system that can fetch theinstructions from the instruction execution system, apparatus, or deviceand execute the instructions associated with logic and/or information.

In the context of this specification, a “non-transitoryprocessor-readable medium” can be any element that can store the programassociated with logic and/or information for use by or in connectionwith the instruction execution system, apparatus, and/or device. Theprocessor-readable medium can be, for example, but is not limited to, anelectronic, magnetic, optical, electromagnetic, infrared, orsemiconductor system, apparatus or device. More specific examples (anon-exhaustive list) of the computer readable medium would include thefollowing: a portable computer diskette (magnetic, compact flash card,secure digital, or the like), a random access memory (RAM), a read-onlymemory (ROM), an erasable programmable read-only memory (EPROM, EEPROM,or Flash memory), a portable compact disc read-only memory (CDROM),digital tape, and other non-transitory media.

The various embodiments described above can be combined to providefurther embodiments. To the extent that they are not inconsistent withthe specific teachings and definitions herein, all of the U.S. patents,U.S. patent application publications, U.S. patent applications, foreignpatents, foreign patent applications and non-patent publicationsreferred to in this specification and/or listed in the Application DataSheet which are owned by Thalmic Labs Inc., including but not limitedto: U.S. Provisional Patent Application Ser. No. 62/167,767; U.S.Provisional Patent Application Ser. No. 62/271,135; U.S. ProvisionalPatent Application Ser. No. 62/017,089; U.S. Provisional PatentApplication Ser. No. 62/053,598; U.S. Provisional Patent ApplicationSer. No. 62/117,316; U.S. Provisional Patent Application Ser. No.62/134,347 (now U.S. Non-Provisional patent application Ser. No.15/070,887); U.S. Provisional Patent Application Ser. No. 62/156,736;U.S. Provisional Patent Application Ser. No. 62/242,844; US PatentPublication No. US 2015-0378164 A1; US Patent Publication No. US2015-0378161 A1; US Patent Publication No. US 2015-0378162 A1; U.S.Non-Provisional patent application Ser. No. 15/145,576; U.S.Non-Provisional patent application Ser. No. 15/145,609; U.S.Non-Provisional patent application Ser. No. 15/145,583; U.S.Non-Provisional patent application Ser. No. 15/046,234; U.S.Non-Provisional patent application Ser. No. 15/046,254; U.S.Non-Provisional patent application Ser. No. 15/046,269; U.S. ProvisionalPatent Application Ser. No. 62/245,792; U.S. Provisional PatentApplication Ser. No. 62/214,600; U.S. Provisional Patent ApplicationSer. No. 62/268,892; U.S. Non-Provisional patent application Ser. No.15/167,472; U.S. Non-Provisional patent application Ser. No. 15/167,458;and U.S. Provisional Patent Application Ser. No. 62/281,041, areincorporated herein by reference, in their entirety. Aspects of theembodiments can be modified, if necessary, to employ systems, circuitsand concepts of the various patents, applications and publications toprovide yet further embodiments.

These and other changes can be made to the embodiments in light of theabove-detailed description. In general, in the following claims, theterms used should not be construed to limit the claims to the specificembodiments disclosed in the specification and the claims, but should beconstrued to include all possible embodiments along with the full scopeof equivalents to which such claims are entitled. Accordingly, theclaims are not limited by the disclosure.

The invention claimed is:
 1. A method of operating a laser projector toproject an image to an eye of a user and to track the eye of the user,the method comprising: outputting visible laser light by a laser module,wherein the laser module includes at least a first laser diode and asecond laser diode, and outputting visible laser light by the lasermodule includes outputting a first visible laser light in a first narrowwaveband from the first laser diode of the laser module and outputting asecond visible laser light in a second narrow waveband from the secondlaser diode, the second narrow waveband different from the first narrowwaveband, wherein the first visible laser light is representative of atleast a first portion of a displayed image and the second visible laserlight is representative of at least a second portion of the displayedimage; controllably and reflectively scanning the first visible laserlight and the second visible laser light by a scan mirror; redirectingthe visible laser light towards the eye of the user by awavelength-multiplexed HOE comprising a first hologram that isresponsive to light in the first narrow waveband and unresponsive tolight that is outside the first narrow waveband and a second hologramthat is responsive to light in the second narrow waveband andunresponsive to light that is outside the second narrow waveband,wherein redirecting the visible laser light towards the eye of the userby the wavelength-multiplexed HOE includes redirecting the first portionof the displayed image towards the eye of the user by the first hologramof the wavelength-multiplexed HOE and redirecting the second portion ofthe displayed image towards the eye of the user by the second hologramof the wavelength-multiplexed HOE, and wherein thewavelength-multiplexed HOE is substantially transparent to environmentallight; detecting a reflection of at least a portion of the visible laserlight from the eye of the user by at least a first narrow wavebandphotodetector and a second narrow waveband photodetector, wherein thefirst narrow waveband photodetector is responsive to light in the firstnarrow waveband and substantially unresponsive to light that is outsideof the first narrow waveband, the second narrow waveband photodetectoris responsive to light in the second narrow waveband and substantiallyunresponsive to light that is outside of the second narrow waveband, andwherein detecting a reflection of the at least a portion of the visiblelaser light from the eye of the user by at least a first narrow wavebandphotodetector and a second narrow waveband photodetector includesdetecting a reflection of the first portion of the displayed image bythe first narrow waveband photodetector and detecting a reflection ofthe second portion of the displayed image from the eye of the user bythe second narrow waveband photodetector; and determining a position ofat least one feature of the eye based on at least the reflection of thefirst portion of the displayed image from the eye of the user detectedby the first narrow waveband photodetector and the reflection of thesecond portion of the displayed image from the eye of the user detectedby the second narrow waveband photodetector.
 2. The method of claim 1,wherein redirecting the visible laser light towards the eye of the userby a wavelength-multiplexed HOE includes converging, by thewavelength-multiplexed HOE, the first visible laser light and the secondvisible laser light towards a first exit pupil at the eye of the user.3. The method of claim 1 wherein: the laser module includes a thirdlaser diode and outputting visible light by a laser module furtherincludes outputting a third visible laser light in a third narrowwaveband by the third laser diode of the laser module, the third narrowwaveband different from both the first narrow waveband and the secondnarrow waveband, and wherein the third visible laser light isrepresentative of at least a third portion of the displayed image;controllably and reflectively scanning the visible laser light by a scanmirror further includes controllably and reflectively scanning the thirdportion of the displayed image by the scan mirror; thewavelength-multiplexed HOE further comprises a third hologram that isresponsive to light in the third narrow waveband and unresponsive tolight that is outside the third narrow waveband and redirecting thevisible laser light towards the eye of the user by thewavelength-multiplexed HOE further includes redirecting the thirdportion of the displayed image towards the eye of the user by the thirdhologram of the wavelength-multiplexed HOE; detecting a reflection of atleast a portion of the visible laser light from the eye of the user byat least a first narrow waveband photodetector further includesdetecting a reflection of the third portion of the displayed image fromthe eye of the user by a third narrow waveband photodetector, whereinthe third narrow waveband photodetector is responsive to light in thethird narrow waveband and substantially unresponsive to light that isoutside of the third narrow waveband; and wherein determining a positionof at least one feature of the eye further includes determining aposition of at least one feature of the eye based on the reflection ofthe third portion of the displayed image from the eye of the userdetected by the third narrow waveband photodetector.
 4. The method ofclaim 3 wherein: the first laser diode is a red laser diode, outputtinga first visible laser light in a first narrow waveband by the firstlaser diode of the laser module includes outputting a red laser light bythe red laser diode, and the first portion of the displayed image is ared portion of the displayed image; the second laser diode is a greenlaser diode, outputting a second visible laser light in a second narrowwaveband by the second laser diode of the laser module includesoutputting a green laser light by the green laser diode, and the secondportion of the displayed image is a green portion of the displayedimage; and the third laser diode is a blue laser diode, outputting athird visible laser light in a third narrow waveband by the third laserdiode of the laser module includes outputting a blue laser light by theblue laser diode, and the third portion of the displayed image is a blueportion of the displayed image.